SLO  Data  Processing  Report 


Demonstration  of  the  Berkeley  UXO  Discrimination  at  Live  Sites 


ESTCP  Project  MR-200838 


February  2010 


Erika  Gasperikova 

Lawrence  Berkeley  National  Laboratory 


This  document  has  been  cleared  for  public  release 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

FEB  2010 

2.  REPORT  TYPE 

N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

Demonstration  of  the  Berkeley  UXO  Discrimination  at  Live  Sites 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Lawrence  Berkeley  National  Laboratory 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

SAR 

18.  NUMBER 
OF  PAGES 

32 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Disclaimer 

This  document  was  prepared  as  an  account  of  work  sponsored  by  the  United  States  Government. 
While  this  document  is  believed  to  contain  correct  information,  neither  the  United  States 
Government  nor  any  agency  thereof,  nor  The  Regents  of  the  University  of  California,  nor  any  of 
their  employees,  makes  any  warranty,  express  or  implied,  or  assumes  any  legal  responsibility  for 
the  accuracy,  completeness,  or  usefulness  of  any  information,  apparatus,  product,  or  process 
disclosed,  or  represents  that  its  use  would  not  infringe  privately  owned  rights.  Reference  herein 
to  any  specific  commercial  product,  process,  or  service  by  its  trade  name,  trademark, 
manufacturer,  or  otherwise,  does  not  necessarily  constitute  or  imply  its  endorsement, 
recommendation,  or  favoring  by  the  United  States  Government  or  any  agency  thereof,  or  The 
Regents  of  the  University  of  California.  The  views  and  opinions  of  authors  expressed  herein  do 
not  necessarily  state  or  reflect  those  of  the  United  States  Government  or  any  agency  thereof  or 
The  Regents  of  the  University  of  California. 

Ernest  Orlando  Lawrence  Berkeley  National  Laboratory  is  an  equal  opportunity  employer. 


2 


TABLE  OF  CONTENTS 


1.  INTRODUCTION . 4 

1.1  Background . 4 

1 .2  Objectives  of  the  Demonstration . 5 

1.2.1  Objectives  of  the  ESTCP  UXO  Discrimination  Study . 5 

1.2.2  ESTCP  Technical  objectives  of  the  Discrimination  Study . 5 

1.2.3  LBNL  Objective . 6 

1.3  Regulatory  Drivers . 6 

2.  TECHNOLOGY  DESCRIPTION . 8 

2.1  Technology  Developmen  t  and  Application . 8 

2.2  Previous  Testing  of  the  Technology . 12 

2.3  Advantages  and  Limitations  of  the  Technology . 12 

3.  DEMONSTRATION  DESIGN . 13 

3.1  Period  of  Operation . 13 

3.2  Scope  of  Demonstration . 14 

4.  DATA  ANALYSIS  AND  INTERPRETATION . 17 

4.1  UXO/scrap  discrimination  approach  using  training  data . 18 

4.2  Application  of  UXO/scrap  discrimination  approach . 24 

5.  PERFORMANCE  ASSESMENT . 28 

6.  ACKNOWLEDGMENTS . 31 

7.  REFERENCES . 31 

8.  ACRONYMS . 32 


3 


1.  INTRODUCTION 


1.1  Background 

In  2003,  the  Defense  Science  Board  observed:  “The  ...  problem  is  that  instruments  that  can 
detect  the  buried  UXOs  also  detect  numerous  scrap  metal  objects  and  other  artifacts,  which  leads 
to  an  enormous  amount  of  expensive  digging.  Typically  100  holes  may  be  dug  before  a  real 
UXO  is  unearthed!  The  Task  Force  assessment  is  that  much  of  this  wasteful  digging  can  be 
eliminated  by  the  use  of  more  advanced  technology  instruments  that  exploit  modern  digital 
processing  and  advanced  multi-mode  sensors  to  achieve  an  improved  level  of  discrimination  of 
scrap  from  UXOs.” 

Significant  progress  has  been  made  in  discrimination  technology.  To  date,  testing  of  these 
approaches  has  been  primarily  limited  to  test  sites  with  only  limited  application  at  live  sites. 
Acceptance  of  discrimination  technologies  requires  demonstration  of  system  capabilities  at  real 
UXO  sites  under  real  world  conditions.  Any  attempt  to  declare  detected  anomalies  to  be 
harmless  and  requiring  no  further  investigation  will  require  demonstration  to  regulators  of  not 
only  individual  technologies,  but  of  an  entire  decision  making  process.  This  characterization 
study  was  be  the  second  phase  in  what  is  expected  to  be  a  continuing  effort  that  will  span  several 
years. 

The  FY06  Defense  Appropriation  contained  funding  for  the  “Development  of  Advanced, 
Sophisticated,  Discrimination  Technologies  for  UXO  Cleanup”  in  the  Environmental  Security 
Technology  Certification  Program  (ESTCP).  ESTCP  responded  by  conducting  a  UXO 
Discrimination  Study  at  the  former  Camp  Sibert,  AL.  The  results  of  this  first  demonstration  were 
very  encouraging.  Although  conditions  were  favorable  at  this  site,  a  single  target  of  interest  (4.2- 
in  mortar)  and  benign  topography  and  geology,  all  of  the  classification  approaches  demonstrated 
were  able  to  correctly  identify  a  sizable  fraction  of  the  anomalies  as  arising  from  non-hazardous 
items  that  could  be  safely  left  in  the  ground.  To  build  upon  the  success  of  the  first  phase  of  this 
study,  ESTCP  sponsored  a  second  study  in  2009  at  the  fonner  Camp  San  Luis  Obispo,  CA,  a  site 
with  more  challenging  topography  and  a  wider  mix  of  targets-of-interest  (TOI). 
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1.2  Objectives  of  the  Demonstration 


1.2.1  Objectives  of  the  ESTCP  UXO  Discrimination  Study 
There  were  two  primary  objectives  of  this  study: 

(1)  Test  and  validate  detection  and  discrimination  capabilities  of  currently  available  and 
emerging  technologies  on  real  sites  under  operational  conditions. 

(2)  Investigate  in  cooperation  with  regulators  and  program  managers  how  discrimination 
technologies  can  be  implemented  in  cleanup  operations. 


1.2.2  ESTCP  Technical  objectives  of  the  Discrimination  Study 

(1)  Test  and  evaluate  capabilities  by  demonstrating  and  evaluating  individual  sensor  and 
discrimination  technologies  and  processes  that  combine  these  technologies.  Compare  advanced 
methods  to  existing  practices  and  validate  the  pilot  technologies  for  the  following: 

(a)  Detection  of  UXOs 

(b)  Identification  of  features  that  distinguish  scrap  and  other  clutter  from  UXO 

(c)  Reduction  of  false  alanns  (items  that  could  be  safely  left  in  the  ground  that  are  incorrectly 
classified  as  UXO)  while  maintaining  Pds  acceptable  to  all 

(d)  Ability  to  identify  sources  of  uncertainty  in  the  discrimination  process  and  to  quantify 
their  impact  to  support  decision  making,  including  issues  such  as  impact  of  data  quality  due 
to  how  data  is  collected 

(e)  Quantify  the  overall  impact  on  risk  arising  from  the  ability  to  clear  more  land  more 
quickly  for  the  same  investment. 

(f)  Include  the  issues  of  a  dig-no  dig  decision  process  and  related  QA/QC  issues 

(2)  Understand  the  applicability  and  limitations  of  the  pilot  technologies  in  the  context  of  project 
objectives,  site  characteristics,  suspected  ordnance  contamination 

(3)  Collect  high-quality,  well-documented  data  to  support  the  next  generation  of  signal 
processing  research. 
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1.2.3  LBNL  Objective 

The  demonstration  objective  was  to  determine  the  discrimination  capabilities  of  the  Berkeley 
UXO  Discriminator  (BUD)  at  San  Luis  Obispo,  CA.  Lawrence  Berkeley  National  Laboratory 
(LBNL)  performed  a  cued  mode  discrimination  survey.  The  data  were  collected  in  accordance 
with  the  overall  study  demonstration  plan,  including  the  system  characterization  with  the 
emplaced  calibration  items  along  a  calibration  line  and  at  a  test  pit. 


1.3  Regulatory  Drivers 

ESTOP  has  assembled  an  Advisory  Group  to  address  the  regulatory,  programmatic  and 
stakeholder  acceptance  issues  associated  with  the  implementation  of  discrimination  in  the 
munitions  response  process. 

Objective  of  Advisory  Group: 

Help  the  ESTOP  Program  Office  explore  a  UXO  discrimination  process  that  will  be  useful  to 
regulators  and  managers  in  making  decisions. 

(1)  Under  what  conditions  would  you  consider  discrimination? 

(2)  What  does  a  pilot  project  need  to  demonstrate  for  the  community  to  consider  not  digging 
every  anomaly  as  a  viable  alternative? 

(a)  Methodology 

(b)  Transparency 

(c)  QA/QC  requirements 

(d)  Validation 

(3)  For  implementation  beyond  the  pilot  project, 

(a)  How  should  proposals  to  implement  discrimination  be  evaluated? 

(b)  Site  suitability 

Geology 

Anomaly  density 
Site  topography 

Level  of  understanding  of  expected  UXO  types 

(c)  Track  record  on  like  sites 

(d)  Perfonnance  on  test  site  or  small  subset  of  site 
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(e)  Understanding  and  management  of  uncertainties 

(4)  Define  data  needs  to  support  decisions,  particularly  with  regard  to  decisions  not  to  dig  all 
detected  anomalies 

(5)  Define  acceptable  end-products  to  support  discrimination  decisions. 

In  support  of  the  above,  provide  input  and  guidance  to  the  ESTCP  Program  Office: 

o  Pilot  project  objectives  and  flow-down  to  metrics 

o  Flow  down  of  program  objectives  to  data  quality  objectives 

o  Demonstration/Data  collection  plans 

o  QA/QC  requirements  and  documentation 

o  Interpretation,  Analysis,  and  Validation 

o  Process  flow  for  discrimination-based  removal  actions 
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2.  TECHNOLOGY  DESCRIPTION 


2.1  Technology  Development  and  Application 

The  ESTCP  has  supported  LBNL  in  the  development  of  the  Berkeley  UXO  Discriminator 
(BUD)  that  not  only  detects  the  object  itself  but  also  quantitatively  determines  its  size,  shape, 
and  orientation.  Furthennore,  BUD  performs  target  characterization  from  a  single  position  of  the 
sensor  platform  above  a  target.  BUD  was  designed  to  detect  UXO  in  the  20  mm  to  155  mm  size 
range  for  depths  between  0  and  1.5  m,  and  to  characterize  them  in  a  depth  range  from  0  to  1.1  m. 
The  system  incorporates  three  orthogonal  transmitters,  and  eight  pairs  of  differenced  receivers. 
The  transmitter-receiver  assembly  together  with  the  acquisition  box,  as  well  as  the  battery  power 
and  global  positioning  system  (GPS)  receiver,  is  mounted  on  a  small  cart  to  assure  system 
mobility.  System  positioning  is  provided  by  state-of-the-art  Real  Time  Kinematic  (RTK)  GPS 
receiver.  The  survey  data  acquired  by  BUD  are  processed  by  software  developed  by  LBNL, 
which  is  efficient  and  simple,  and  can  be  operated  by  relatively  untrained  personnel.  BUD  is 
shown  in  Figure  1. 


Figure  1.  Berkeley  UXO  Discriminator  (BUD) 
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The  BUD  employs  three  orthogonal  transmitters  and  eight  pairs  of  differenced  receivers.  Eight 
receiver  coils  are  placed  horizontally  along  the  two  diagonals  of  the  upper  and  lower  planes  of 
the  two  horizontal  transmitter  loops.  These  receiver  coil  pairs  are  located  on  symmetry  lines 
through  the  center  of  the  system  and  each  pair  sees  identical  fields  during  the  on-time  of  current 
pulses  in  the  transmitter  coils.  They  are  wired  in  opposition  to  produce  zero  output  during  the 
on-time  of  the  pulses  in  three  orthogonal  transmitters.  This  configuration  dramatically  reduces 
noise  in  measurements  by  canceling  background  electromagnetic  fields  (these  fields  are  uniform 
over  the  scale  of  the  receiver  array  and  are  consequently  nulled  by  the  differencing  operation), 
and  by  canceling  noise  contributed  by  the  tilt  of  the  receivers  in  the  Earth’s  magnetic  field,  and 
greatly  enhances  receivers’  sensitivity  to  gradients  of  the  target  response. 

Data  acquisition  is  perfonned  on  a  single  board.  The  transmitter  coils  are  powered  separately 
from  the  data  acquisition  board.  Pulsers  provide  resonant  circuit  switching  to  create  bi-polar  half¬ 
sine  pulses  of  350  ps  width.  The  current  has  a  -18  A  peak  and  a  resonant  circuit  voltage  of  -750 
Volts.  The  operational  overall  half-sine  duty  cycle  is  -12%.  The  resonant  frequency  of  the 
inductive  load  is  -90  kHz.  Transients  are  digitized  with  a  sampling  interval  of  4  ps.  The  sensors 
are  critically  damped  6-inch  325  turn  loops  with  a  self-resonant  frequency  of  25  kHz.  The  data 
acquisition  board  has  12  high-speed  ADC  channels  for  output.  Eight  of  these  channels  are  used 
for  the  signal  from  receiver  coils,  and  the  remaining  four  channels  provide  information  about  the 
system  (i.e.  tilt  information,  time  stamps,  transmitter  current). 

It  has  been  demonstrated  that  a  satisfactory  classification  scheme  is  one  that  detennines  the 
principal  dipole  polarizabilities  of  a  target  -  a  near  intact  UXO  displays  a  single  major 
polarizability  coincident  with  the  long  axis  of  the  object  and  two  equal  transverse  polarizabilities. 
The  induced  moment  of  a  target  depends  on  the  strength  of  the  transmitted  inducing  field.  The 
moment  normalized  by  the  inducing  field  is  the  polarizability.  This  description  of  the  inherent 
polarizabilities  of  a  target  is  a  key  in  discriminating  UXO  from  irregular  scrap  metal.  Figures  2-4 
illustrate  a  discrimination  capability  of  the  system  for  UXO  objects  (Figures  2  and  3),  and  a 
scrap  metal  (Figure  4).  All  three  figures  have  estimated  principal  polarizabilities  as  a  function  of 
time  plotted  on  the  left,  values  of  true  and  estimated  location  and  orientation  on  the  right,  and 
object  images  at  the  bottom.  While  UXO  objects  have  a  single  major  polarizability  coincident 
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with  the  long  axis  of  the  object  and  two  equal  transverse  polarizabilities  (Figure  2-3),  the  scrap 
metal  exhibits  three  distinct  principal  polarizabilities  (Figure  4).  The  locations  and  orientations 
are  recovered  within  a  few  percent  of  true  values  for  all  three  objects.  Moreover,  UXO  have 
unique  polarizability  signatures,  and  thus  distinctions  can  be  made  among  various  UXOs. 


Figure  2.  Inversion  results  for  the  principal  polarizabilities,  location  and  orientation  of  81  mm 
M821A1  projectile 


Figure  3.  Inversion  results  for  the  principal  polarizabilities,  location  and  orientation  of  105  mm 
M60  projectile 


10 


Figure  4.  Inversion  results  for  the  principal  polarizabilities,  location  and  orientation  of  19x8  cm 
scrap  metal 

The  detection  performance  of  the  system  is  governed  by  an  object’s  size-depth  relationship.  If 
we  assume  that  BUD’s  lower  receiver  plane  is  0.2  m  above  the  ground,  the  system  can  detect  and 
discriminate  objects  with  depth  uncertainty  of  10%  down  to  1.3  m  and  0.9  m,  respectively.  Any 
objects  buried  at  the  depth  more  than  1.3  m  and  0.9  m  will  have  a  low  probability  of  detection 
and  discrimination,  respectively. 

Object  orientation  estimates  and  equivalent  dipole  polarizability  estimates  used  for  large  and 
shallow  UXO/scrap  discrimination  are  more  problematic  as  they  are  affected  by  higher  order 
(non-dipole)  terms  induced  in  objects  due  to  source  field  gradients  along  the  length  of  the 
objects.  For  example,  a  vertical  0.4  m  object  directly  below  the  system  needs  to  be  about  0.90  m 
deep  for  perturbations  due  to  gradients  along  the  length  of  the  object  to  be  of  the  order  of  20  % 
of  the  uniform  field  object  response.  Similarly,  vertical  objects  0.5  m,  and  0.6  m  long  need  to  be 
1.15  m,  and  1.42  m,  respectively,  below  the  system.  For  horizontal  objects  the  effect  of  gradients 
across  the  objects’  diameter  are  much  smaller.  For  example,  155  mm  and  105  mm  projectiles 
need  to  be  only  0.30  m,  and  0.19  m,  respectively,  below  the  system.  A  polarizability  index  (in 
cm  ),  which  is  an  average  value  of  the  product  of  time  (in  seconds)  and  polarizability  rate  (in 
m  /s)  over  the  35  sample  times  logarithmically  spaced  from  140  to  1400  ps,  and  three 
polarizabilities,  can  be  calculated  for  any  object.  In  this  survey,  we  used  this  polarizability  index 
to  decide  when  the  object  is  in  a  uniform  source  field.  Objects  with  the  polarizability  index 
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3  3 

smaller  than  600  cm  and  deeper  than  1.8  m  below  BUD,  or  smaller  than  200  cm  and  deeper 
than  1.35  m,  or  smaller  than  80  cm  and  deeper  than  0.90  m,  or  smaller  than  9  cm  and  deeper 
than  0.20  m  below  BUD  are  sufficiently  deep  that  the  effects  of  vertical  source  field  gradients 
should  be  less  than  15%  and  are  not  expected  to  be  a  problem.  All  other  objects  are  considered 
large  and  shallow  objects.  For  the  object  characterization,  UXO/scrap  discrimination, 
measurements  at  each  flag  were  taken  on  a  grid  as  shown  in  Figure  7.  This  allows  the  system  get 
further  away  from  the  object,  and  hence  minimizes  source  gradients,  at  one  or  more  locations  in  the 
case  that  a  large  shallow  object  is  found. 


2.2  Previous  Testing  of  the  Technology 

The  perfonnance  of  the  BUD  has  been  demonstrated  at  our  local  test  site  in  California,  as  well  as 
the  Calibration  and  Blind  Test  Grids  and  the  Open  Field  Range  at  the  Yuma  Proving  Ground 
(YPG),  AZ,  Camp  Sibert,  AL,  and  FE  Warren  AFB,  WY.  The  results  have  been  presented  at 
various  meetings  and  published  in  scientific  journals. 


2.3  Advantages  and  Limitations  of  the  Technology 

This  system  not  only  detects  but  also  discriminates  UXO  from  non-UXO/scrap  and  gives  its 
characteristics  (location,  size,  polarizability,  aspect  ratio)  from  a  single  position  of  the  sensor 
platform  above  the  object.  BUD  was  designed  to  detect  and  discriminate  UXO  in  the  20  mm  to 
155  mm  size  range  buried  anywhere  from  0  to  1.5  m  depth  and  from  0  to  1.1  m,  respectively.  Any 
objects  buried  at  the  depth  more  than  1.5  m  will  have  a  low  probability  of  detection,  and  any 
objects  buried  at  the  depth  more  than  1.1m  will  have  a  low  probability  of  discrimination.  With 
existing  algorithms  in  the  system  computer  it  is  not  possible  to  recover  the  principal 
polarizabilities  of  large  objects  close  to  the  system.  Detection  of  large  shallow  objects  is  assured, 
but  at  present  discrimination  is  not.  Post  processing  of  the  field  data  is  required  for  shape 
discrimination  of  large  shallow  targets.  See  Chapter  2.1  for  details. 
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3.  DEMONSTRATION  DESIGN 


3.1  Period  of  Operation 

The  BUD  cued  discrimination  survey  at  San  Luis  Obispo  took  place  between  June  22,  2009  and 
July  9,  2009.  Data  were  collected  over  244  flags  in  the  south  grid  and  295  flags  in  the  east  grid 
shown  in  yellow  in  Figure  5.  The  gray  area  in  Figure  5  indicates  the  location  of  training  data  for 
our  discrimination  analysis.  The  ESTCP  office  provided  the  locations  of  these  flags.  Data 
acquisition  took  13  working  days.  We  had  no  down  time  with  the  system,  and  finished  the  survey 
ahead  of  schedule.  The  field  crew  consisted  of  three  people,  and  the  PI  was  present  a  part  of  the 
survey. 


Figure  5:  BUD  cued  survey  (in  yellow)  location.  Training  data  locations  are  shown  in  gray. 
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3.2  Scope  of  Demonstration 

The  ESTCP  UXO  Discrimination  Demonstration  Site  was  located  at  former  Camp  San  Luis 
Obispo,  CA.  LBNL  performed  a  discrimination  survey  of  the  calibration  line,  test  pit,  and 
targets/locations  (flags)  provided  to  us  by  the  ESTCP  office. 

The  calibration  line  contained  four  types  of  UXOs  that  were  expected  at  the  site  -  60  mm,  8 1 
mm,  and  4.2-in  mortars,  and  2.35-in  rocket.  Typical  polarizability  responses  of  these  objects  are 
shown  in  Figure  6.  Each  plot  contains  major  (solid  line),  medium  (dashed  line)  and  minimum 
(dotted  line)  polarizabilities  as  a  function  of  time  and  a  picture  of  the  item. 


to'  F 


a 


60  mm  mortar 


— —  maximum 
-  -medium 
minimum 
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2.36-in  rocket 


Figure  6:  Polarizability  responses  as  a  function  of  time  for  (a)  60  mm  mortar,  (b)  2-36  in  rocket, 
(c)  81  mm  mortar,  and  (d)  4.2-in  mortar. 


The  BUD  was  operated  in  the  cued  mode.  BUD  was  brought  to  marked  locations  and  run  in  the 
characterization/discrimination  mode.  The  three  discriminating  polarizability  responses  were 
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recorded  and  visually  presented  on  the  computer  screen.  The  depth  and  horizontal  location  with 
respect  to  the  cart  was  recorded,  together  with  a  GPS  location  of  the  reference  point  on  the  cart, 
both  corrected  for  tilt  due  to  a  steep  terrain,  using  single  object  inversion  algorithm.  As  described 
in  Chapter  2.1,  object  orientation  estimates  and  equivalent  dipole  polarizability  estimates  used 
for  large  and  shallow  UXO/scrap  discrimination  are  more  problematic  as  they  are  affected  by 
higher  order  (non-dipole)  terms  induced  in  objects  due  to  source  field  gradients  along  the  length 
of  the  objects.  Therefore,  stationary  data  were  collected  on  11 -point  grid  around  each  flag  as 
shown  in  Figure  7.  This  allowed  the  system  get  further  away  from  the  object,  and  hence 
minimized  source  gradients,  at  one  or  more  locations  in  the  case  that  a  large  shallow  object  was 
found.  The  orientation  of  our  short  profiles  was  chosen  based  on  topography,  so  that  we  didn’t 
have  to  push/hold  BUD  up  or  down  the  hill;  when  possible  it  was  along  topography  contours. 
We  built  a  lightweight  template  that  was  easy  to  carry,  and  the  measurements  points  were 
painted  on  the  ground  with  an  environmentally  friendly  paint.  BUD  then  occupied  each  of  the  1 1 
points  and  acquired  a  stationary  measurement. 

The  cart  was  equipped  with  a  two-component  tiltmeter  and  three-component  magnetometer. 
Both  devices  were  calibrated  before  the  survey.  The  tiltmeters  readings  were  used  to  compute  the 
cart  pitch  (positive  pitch  =  cart  front  down  =  negative  tiltmeter  change),  and  roll  (positive=cart 
rightside  down  -  positive  tiltmeter  change).  The  magnetometer  values  were  used  to  compute 
vector  magnetic  field  relative  to  cart  coordinates.  This  was  rotated  into  leveled  cart  coordinates, 
using  the  cart  pitch  and  roll  measurements.  Knowing  the  GPS  antenna  height  above  the  BUD 
lower  cube  surface,  cart  azimuth,  tilt,  and  roll,  the  offset  of  the  antenna  from  the  cube  coordinate 
origin  (at  cube  bottom  center)  was  calculated  and  subtracted  from  the  reported  GPS  coordinates. 
As  an  indicator  of  the  accuracy  of  the  computed  cart  azimuth,  the  sum  of  squared  differences  in 
horizontal  magnitude  and  vertical  components  of  magnetic  field  from  their  average  values  was 
computed  and  normalized  by  the  average  squared  magnetic  field  magnitude.  Values  greater  than 
2xl0"4  indicate  a  likely  error  in  computed  orientation  greater  than  0.8  degrees,  and  as  much  as  1 
cm  mislocation  for  points  at  the  corners  of  the  1  m  footprint  of  the  device. 
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Figure  7:  BUD  template  for  1 1  measurements  around  the  flag 
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4.  DATA  ANALYSIS  AND  INTERPRETATION 


The  first  step  prior  to  data  collection  was  a  system  calibration  and  a  background  level  estimation. 
We  measured  the  background  noise  with  transmitters  in  off-position  and  calculated  spectrum  of 
all  channels.  Then  we  turned  the  transmitters  on  and  measured  the  background  field  on  all 
channels.  This  was  repeated  at  least  twice  to  make  sure  the  background  field  was  stable  and 
could  be  used  as  the  baseline  measurement  that  was  subtracted  from  the  data. 

Twelve  channels  of  field  data  were  recorded  at  a  rate  of  250  k-samples/second  for  each  of  three 
transmitters.  Field  data  were  stacked  together  in  a  field  programmable  gate  array  (FPGA)  and 
transferred  to  a  field  computer  (laptop)  fonning  a  primitive  stack,  labeled  with  header  infonnation 
(instrument  position,  tilt  and  heading,  time  stamp,  channels  of  transmitter  current,  etc).  An  even 
number  of  primitive  stacks  was  averaged  together  to  form  stacked  data  for  further  processing.  The 
peak  transmitter  current  was  estimated  from  the  stacked  transmitter  current  channel  record,  and  the 
data  were  nonnalized  by  that  value.  Nominal  transmitter  shut-off  time  was  estimated,  and 
induction  responses  were  computed  at  35  logarithmically  spaced  times  between  140  and  1400  ps, 
averaged  in  half-sine  windows  with  widths  10%  the  center  time  after  transmitter  pulse  shut-off. 
Responses  were  differenced  with  background  responses  collected  over  a  nearby  site  detennined  to 
be  relatively  free  of  metallic  objects  by  having  a  system  response  that  varied  little  with  system 
translation.  The  resulting  24  channels  of  nonnalized  responses  were  then  inverted  for  candidate 
object  position  and  principal  polarizabilities  as  a  function  of  time  after  transmitter  shut-off.  Data 
before  140  ps  were  ignored.  Polarizability  plots  for  candidate  objects  were  examined  for 
consistency  with  an  inductive  object  response. 

Both  field  data  and  inversion  results  have  been  archived.  The  data  are  retained  on  a  portable 
external  disk,  and  the  inversion  results  are  retained  both  on  the  portable  external  disk  and  CD- 
ROM  for  archiving  and  distribution. 
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4.1  UXO/scrap  discrimination  approach  using  training  data 


Characterization  of  UXO  and  clutter  was  done  using  a  training  data  set  that  was  provided  by  the 
ESTCP  office  (gray  area  in  Figure  5),  those  collected  over  the  test  pit  at  the  site,  and  data 
collected  over  the  targets  of  interest  in  previous  surveys.  The  training  data  consisted  of  sets  of 
principal  polarizability  responses  at  35  logarithmically  spaced  times  centered  from  140  ps  to 
1400  ps  after  transmitter  shut-off.  The  training  data  set  contained  principal  polarizability 
responses  of  312  objects  -  108  UXO,  and  204  pieces  of  scrap  metal.  113  objects  were  from  San 
Luis  Obispo,  and  199  from  our  previous  library.  The  approach  in  this  study  was  similar  to  the 
one  applied  to  the  fonner  Camp  Sibert  data  (Gasperikova  et  ah,  2009).  However,  in  this  survey, 
the  level  of  difficulty  was  increased  by  presence  of  multiple  objects,  and  we  present  a  new  way 
how  to  analyze  these  kinds  of  data. 

The  data  time  interval  was  subdivided  logarithmically  into  ndiv  =  3  subintervals.  The  product  of 
each  principal  polarizability  with  its  sample  time  was  averaged  over  each  of  these  intervals. 
Since  there  are  three  principal  polarizabilities,  this  results  in  nfeat  =  3n(iiv  reduced  data,  henceforth 
called  “features.”  We  used  an  additional  feature,  the  logarithm  of  the  vector  magnitude  of  the 
other  features  (in  m3),  which  increases  the  total  number  of  features  nfeat  to  3n<jiv+l,  in  this  case 
ten.  When  this  feature  was  appended,  the  partial  vector  of  other  features  was  rescaled  to  have 
unit  magnitude.  For  each  training  response  or  response  to  be  classified,  an  nfeat  dimensional 
vector  was  constructed  from  the  features  of  responses.  Each  of  the  features  had  its  median  and 
median  absolute  deviation  (MAD)  computed  separately  for  UXO  and  scrap  training  data. 

In  cross  validation,  results  from  a  subset  of  training  data  were  used  to  predict  something  about 
the  remaining  training  data.  This  was  done  many  times  (“repeats”),  excluding  a  different  set  of 
training  data  each  time,  and  then  a  choice  was  made  based  on  what  gave  the  best  predictions 
averaged  over  many  repeats.  For  the  UXO  versus  scrap  discrimination  problem,  average  cross- 
validated  estimated  probability  of  UXO  being  scrap  was  0.052,  when  the  number  of  subintervals 
(n^v)  was  3  (nfeat  =10  features)  and  all  training  data  were  used.  When  only  training  data  from 
San  Luis  Obispo  were  used  the  average  cross-validated  estimated  probability  of  UXO  being 
scrap  was  0.25.  In  this  work,  a  constant  number  of  UXO  and  scrap  training  data  were  excluded  at 
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a  time,  in  roughly  equal  proportions.  The  number  of  the  excluded  data  was  chosen  so  that  one 
response  was  withheld  at  a  time  from  the  smaller  of  the  sets  of  UXO  and  scrap  training  data.  The 
set  of  excluded  data  was  cycled  through  excluding  each  UXO  response  once  and  most  scrap 
responses  once. 

For  convenience,  features  were  differenced  with  median  values  (for  UXO  or  scrap)  and 
nonnalized  by  feature  MADs  (for  UXO  or  scrap),  separately  for  consideration  as  UXO  or  scrap, 
and  denoted  by  Vi<uxo)  or  Vi(scrap)  for  the  two  normalizations,  respectively.  Training  data  from  UXO 
and  scrap  classes  were  used  to  form  trimmed-feature  covariance  matrices  for  the  two  classes 
separately: 


C 


(class) 


1 


(class) 


n 


Z  (class)  /  (class)  x  t  , 

Vi  (V;  )  + 
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i  in  class,!  v/class^  I  >median 
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where  (class)  is  either  (uxo)  or  (scrap),  t  denotes  transpose,  and  n(class)  is  the  number  of  (class) 
responses.  In  the  second  sum,  the  contribution  of  large  magnitude  feature  vectors  are 
downweighted.  After  forming  C(class),  it  was  biased  by  dividing  its  off-diagonal  elements  by  1+  U 
(n(dass))-i/2,  damping  its  smaller  eigenvalues.  A  feature  vector  Vi(class)  probability  density  function 
was  estimated  empirically  as 
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where  T  is  the  gamma  function,  and  p^as)  >2. 
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Equation  (2)  is  a  generalization  of  a  Cauchy  distribution.  As  p^s)  approaches  infinity  the 
distribution  approaches  a  Gaussian  distribution.  With  parameter  p^y  =  3  the  outer  exponent 
has  the  smallest  half-integer  value  for  which  distribution  has  a  finite  variance.  This  value  allows 
for  very  heavy  tailed  distributions.  Empirical  probability  densities  were  estimated  separately  for 
UXO  and  scrap  classes. 

In  cross-validation,  densities  in  equation  (2)  were  computed  for  UXO  and  scrap  classes  from 
non-excluded  responses,  and  feature  vectors  v  -uxo)  and  v  -scrap)  were  computed  for  excluded 

training  responses,  where  for  j’th  response,  the  two  differ  in  component  offsets  and 
nonnalization.  The  first  was  used  in  estimating  the  response’s  likelihood  as  a  UXO  response,  and 
the  second  in  estimating  its  likelihood  as  a  scrap  response. 


Allowing  for  unequal  a  priori  probability  of  a  response  being  due  to  UXO  or  scrap,  with  their 
ratio  being  a2,  the  probability  that  the  response  is  due  to  scrap  is: 


?  (scrap)  /  (scrap) 


(scrap)  /  (scrap)  \  _ 


(y  (scrap)  )  =  ■ 


^  v  (scrap)  ^ 


^2^*(uxo)  ^y(uxo)  ^  ^  (scrap)  ^  (scrap)  ^ 

(Bayes’  rule,  e.g.,  Hoel  et  ah,  1971)  and  the  probability  that  the  response  is  due  to  UXO  is 

f (scrap)  (^T  (scrap) 


(4) 


*(ux°)  <yux°) )  =  i — - 


)  (  y  (scrap)  ^ 


(uxo)  ^y  (uxo)  ^  £  (scrap)  ^  (scrap)  ^ 


(5) 


Probabilities  in  Equations  (4)  or  (5)  form  the  basis  for  the  discrimination  between  UXO  and 
scrap  classes.  The  class  feature  covariance  matrices  C(uxo)  and  C(scrap),  and  the  densities  f 11X01  and 
fiscrap)  were  compUted  from  subsets  of  the  training  data.  UXO  and  scrap  probabilities  -  equations 
(4)  and  (5)  -  were  estimated  for  remaining  (excluded)  training  data.  Estimated  UXO  probabilities 
of  excluded  training  data  were  summed  over  many  repeats. 

In  practice,  we  choose  parameter  a"  empirically.  For  a  given  value  of  a  ,  the  scrap  probabilities 
of  all  training  data  are  summed  as 

^n(scrap)^  _  ^p(scrap)(v(scrap)^  ^ 
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where  the  densities  entering  sum  (6)  are  evaluated  as  before,  excluding  the  training  data  for 
which  a  density  is  being  calculated  from  the  covariance  matrix  calculations  and  the  sums  in 
densities  in  equation  (2).  Since  the  number  of  scrap  responses  in  the  training  data  set  is  known 
parameter  a"  is  adjusted,  so  that 

<n(scrap)>  _  n(scrap)  /y\ 

2  2 

Quantity  (6)  is  monotonic  in  a",  so  solution  is  unique.  Newton's  method  started  from  a  =  1, 

keeping  a"  from  decreasing  to  less  than  0.1  of  its  previous  value  on  any  iteration,  works  very 
well.  Since  p(scrap)  =  l-p<ux0),  this  criterion  for  setting  a2  also  sets  the  sum  of  p(UXO)(vj|UXO))  to  the 
number  of  UXO  responses  in  the  training  data  set. 


After  computing  fuxo,(vj(uxo))  and  t<sc'apl(vj(NC'apl)  for  the  set  of  excluded  responses,  the  set  of 
excluded  responses  was  changed,  trimmed  feature  covariance  matrices  recomputed,  and  densities 
computed  for  the  set  of  new  excluded  responses.  n^"ap)  with  unreserved  scrap  training  responses 

and  ncyci  x  n(^X0)  with  unreserved  UXO  training  responses  are  cycled  through  the  excluded  data 
sets.  Parameter  pj,al“s)  in  equation  (2)  was  chosen  separately  for  UXO  and  scrap  classes,  in  each 
case  using  the  integer  maximizing  the  log  likelihood 


jU class)  =  ^  log(p(ClaSS>  (y(.class)  )) 

j  in  class 

of  the  training  data  of  the  corresponding  class. 


(8) 


In  summary,  the  UXO  and  scrap  data  were  randomly  ordered  within  the  class.  A  sequence  of 
candidate  p^lay  values  were  cycled  through  as  an  outer  loop.  Sets  of  excluded  UXO  and  scrap 
training  data  were  chosen  starting  with  the  first  on  their  randomly  ordered  lists.  Trimmed  feature 
covariance  matrices  were  computed  excluding  these.  Quantities  f<uxo)(Vj(uxo))  and  f*scrap)(vj|scrap)) 
were  computed  for  the  excluded  data,  and  the  sum  of  their  logarithms  evaluated  at  the  training 
points  accumulated.  The  sets  of  excluded  responses  were  changed  (moving  down  the  random 
ordered  lists),  trimmed  feature  covariance  matrices  recomputed,  and  densities  computed  for  the 
new  set  of  excluded  responses,  based  on  the  non-excluded  responses.  After  cycling  the  training 
data  through  the  excluded  sets,  and  cycling  through  the  outer  loop  of  prospective  p^as) ,  p^lajs| 
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was  chosen  for  each  of  scrap  and  UXO  distributions  to  maximize  the  likelihood  of  the  training 
data. 


Once  p^tfa0)  and  p[,sxc™p)  was  selected,  covariance  matrices  C(UXO)  and  C<scrap)  were  recomputed 

using  all  training  data  to  compute  feature  medians  and  MADs  for  UXO  and  for  scrap  responses, 
and  in  forming  the  covariance  matrices  themselves.  Similarly,  all  training  data  were  then  used  in 
reforming  empirical  distributions  for  scrap  and  UXO  classes  analogously  to  equations  (2)  and  (3) 
but  summed  over  all  responses,  omitting  any  self-response  terms  (i  =  j).  The  resulting  covariance 
estimates,  and  empirical  probability  distributions  for  v(scrap)  and  v(UXO)  were  then  used  to  evaluate 
the  probability  that  a  response  is  due  to  scrap  or  UXO  through  equation  (4)  using  the  response's 
feature  vector,  shifted  and  nonnalized. 

After  this  calibration,  the  algorithm  was  applied  to  the  set  of  unknown  responses,  and  the 
discrimination  between  UXO  and  scrap  classes  was  conducted  using  equation  (4)  and  (5). 
Furthermore,  the  priority  dig  list  was  constructed  in  a  following  way:  (1)  each  cued  location/flag 
was  assigned  (2)  a  probability  of  being  a  clutter  (least  dangerous  items  first,  with  a  probability 
equal  to  one),  (3)  a  rank  (integer  number  starting  with  1,  and  for  the  category  4  entered  -9999), 
(4)  a  category  (1,  2,  3,  or  4  explained  below),  and  (5)  overlap  status  (zero  for  no  overlap,  non¬ 
zero  for  overlapping  signatures). 

The  four  categories  were: 

( 1)  Can  Analyze  -  Likely  Clutter 

(2)  Can  Analyze  -  Can’t  Decide 

(3)  Can  Analyze  -  Likely  Munition 

(4)  Cannot  Analyze 

The  cumulative  number  of  anomalies  with  p(UXO)  below  a  given  value  is  plotted  in  Figure  8  for 
TOI  (solid)  and  scrap  (dashed)  in  the  SLO  training  data.  Based  on  the  training  data,  all  objects 
with  p0|uxo)  smaller  than  0.0005  were  considered  as  clutter.  In  another  words,  the  boundary 
between  category  1  and  2  was  taken  at  0.9995  probability  of  being  scrap.  The  boundary  between 
category  2  and  3  was  taken  at  0.25  probability  of  being  UXO.  Vertical  lines  in  Figure  8  indicate 
these  boundaries. 
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cumulative  distribution  SLO  training  responses 


Figure  8:  Estimated  probabilities  of  object  being  scrap  or  UXO  for  SLO  training  data  set. 

For  the  purposes  of  this  plot  as  well  as  in  creating  density  functions  using  equation  (2),  object 
under  flag  1260  was  treated  as  scrap  (it  has  p(UXO)  =  0.000003].  With  these  cutoffs  there  were  28 
objects  in  category  1,  23  in  category  2,  one  in  category  3,  and  three  in  category  4  for  the  SLO 
scrap  training  data.  Similarly,  there  were  zero  objects  in  category  1,  one  in  category  2,  and  eight 
in  category  3  for  the  SLO  TOI  training  data.  In  this  training  data  set  analysis  we  correctly 
identified  44%  of  total  number  of  objects  that  can  be  left  in  the  ground  (scrap),  while  no  TOI  was 
left  in  the  ground.  If  object  1260  would  be  considered  as  TOI,  in  this  analysis,  we  would  have 
one  false  negative.  Category  4  was  defined  as  (a)  measurements  at  the  sites  with  overlapping 
signatures  (three  adjacent  anomalies  with  maxima  closer  than  3  m),  and  (b)  those  which 
inversions  have  nns  relative  residuals  greater  than  0.5. 

Scrap  and  UXO  probability  density  functions  were  estimated  at  the  different  data  point  values. 
Using  the  maximum  of  likelihood  functions  (8)  over  integer  values  of  paxxa0)  and  p^ap)  for  the 

UXO  and  scrap,  probability  densities  are  at  p‘xXO)  =  9  and  p -sx“ap)  =  23,  so  these  values  were 
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selected.  To  satisfy  criterion  that  the  summed  UXO  probabilities  should  yield  the  true  number  of 
UXO  in  the  training  set,  a2  =  15242  was  similarly  selected. 


4.2  Application  of  VXO/scrap  discrimination  approach 

In  applying  the  method  to  data  from  San  Luis  Obispo,  we  allowed  for  the  possibility  of  multiple 
objects  close  to  a  cued  flag.  To  give  sufficient  data  for  distinguishing  multiple  objects,  for  each 
provided  location/flag  we  collected  1 1  measurements  using  a  scheme  shown  in  Figure  7. 
Measurements  were  repeated  when  GPS  accuracy  error  estimate  was  greater  than  0.02  m.  When 
some  sets  of  measurements  were  repeated,  the  set  with  overall  lower  GPS  error  was  used,  but 
individual  measurements  from  other  sets  included  when  their  GPS  error  was  lower.  When 
measurements  from  several  sets  were  mixed,  locations  with  multiple  measurements  were 
weighted  approximately  inversely  with  respect  to  GPS  error  value.  The  data  were  inverted  for 
dipole(s)  position  and  principal  polarizability  curves  as  a  function  of  time  using  the  empirical 
distribution  evolutionary  algorithm  (e.g.,  Smith  and  Morrison,  2004,  2005)  using  multiple  site 
measurements  and  multiple  dipole  positions,  with  dipole  positions  and  polarizabilities  as  free 
parameters. 

For  uncorrelated  Gaussian  noise,  one  expects  a  slight  improvement  of  data  fit  with  increased 
number  of  dipoles  fitted  to  the  data.  For  uncorrelated  Gaussian  noise  added  to  data  from  objects 
well  approximated  by  single  dipole  polarizabilities,  one  expects  larger  improvements  in  data 
misfit  with  increased  number  of  dipoles,  up  to  the  true  number  of  objects,  and  slight 
improvements  with  additional  dipoles.  In  our  case,  the  data  had  uncertainties  in  instrument 
location  between  different  measurements  on  the  order  of  2  cm.  Uncertainty  in  cart  position  is 
equivalent  to  correlated  error  in  the  measurements  due  to  differences  between  what  is  measured 
at  the  actual  location,  and  what  would  have  been  measured  at  the  estimated  position,  and  are 
roughly  proportional  to  the  measured  data.  We  approximated  this  by  including  an  additive  5% 
relative  noise  in  the  data  errors  used  inversely  to  weight  the  data  (by  adding  the  square  of  5%  of 
the  data  magnitudes  to  the  estimated  data  variances).  Instrument  mislocation  is  correlated 
between  measurements  at  a  single  system  location,  and  much  less  correlated  between  different 
system  locations  used  in  an  inversion. 
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Because  of  the  presence  of  instrument  location  errors,  which  act  as  correlated  noise,  the 
appropriate  cut-off  levels  of  a  misfit  ratio  between  one  and  two  dipole  inversions,  and  between 
two  and  three  dipole  inversions  was  determined  from  examination  of  their  distributions.  The 
amount  of  misfit  reduction  between  fitting  one-  or  two-dipoles,  and  between  fitting  two-  or 
three-dipoles,  for  inversions  of  data  sets  in  the  SLO  south  grid  area,  is  plotted  as  a  scatter  plot  in 
Figure  9. 


resid  3  dip  /  resid  2  dip 

Figure  9:  Scatter  plot  of  a  misfit  between  three-  and  two-dipole  inversions  and  two-  and  one- 
dipole  inversions. 

A  small  gap  in  the  ratio  of  three  to  two-dipole  inversion  misfits  was  noted  near  0.8  in  the  scatter 
plot.  A  similar  small  gap  in  the  ratio  of  two  to  one-dipole  inversion  misfits  was  noted  near  0.8. 
So,  0.8  was  used  as  the  separation  point  for  misfit  improvements  between  two  and  three-dipole 
inversions;  when  this  misfit  ratio  was  less  than  0.8,  the  data  were  considered  to  require  a  three 
dipole  (three  object)  interpretation,  and  when  greater  than  0.8  a  two  or  one  object  interpretation. 

Data  with  that  ratio  greater  than  0.8,  were  similarly  divided  into  two  and  one  object 
interpretations  based  on  the  ratio  of  misfits  of  two  and  one-dipole  inversions,  with  values  of  this 
ratio  less  than  0.8  interpreted  with  two  dipoles,  and  greater  than  0.8  with  one  dipole.  If  more  than 
three  objects  are  present,  interpretation  with  the  three-dipole  model  may  miss  an  object.  To  guard 
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against  this,  the  rms  residual  relative  to  the  data  at  the  location  closest  to  the  anomaly  maximum 
was  considered.  Any  data  with  relative  residuals  greater  than  0.5  was  considered  inadequately  fit. 
Examples  of  inadequate  inversions  that  would  be  classified  as  the  category  4  -  Cannot  Analyze 
are  data  from  the  training  data  set  for  flags  1276,  1292,  and  1312,  which  are  all  scrap  responses. 
Among  targets  of  interest  (TOI)  in  the  training  data  set,  six  were  fitted  with  one  dipole,  one  with 
two  dipoles,  and  three  with  three  dipoles.  Object  under  flag  1260  was  omitted  from  the  TOI  list, 
because  of  its  asymmetric  response.  Other  scrap  training  data  responses  were  fitted  as  follows  - 
15  with  one  dipole,  21  with  two  dipoles  and  16  with  three  dipoles.  In  training  TOI  multi-dipole 
inversions,  one  object  (flag  1253)  was  split  into  two  dipoles  separated  0.24  m  with  principal 
polarizabilities  displaying  symmetry  typical  of  UXO,  so  both  were  included  as  training  data.  In 
the  three  TOI  three-dipole  inversions,  each  had  one  dipole  with  symmetric  principal 
polarizabilities,  which  was  used  as  a  training  response,  and  the  other  two  dipoles  were  not  used 
in  training  or  classification.  The  different  dipoles  from  the  multi-dipole  scrap  inversions  were 
each  used  as  scrap  training  responses,  except  three  dipoles  which  were  further  than  1  m  laterally 
from  the  nearest  sounding  and  were  omitted  (one  each  at  flags  759,  858,  and  1301).  Additionally, 
anomalies  have  been  classified  by  the  number  of  adjacent  anomalies  with  maxima  closer  than  3 
m.  All  the  anomalies  with  more  that  three  adjacent  anomalies  with  maxima  closer  than  3  m  were 
classified  as  having  overlapping  signatures,  and  as  category  4. 

Following  the  process  described  above,  the  median  normalized  time  polarizability  products  for 
the  three  binned  sample  times  centered  at  0.2135,  0.4341,  and  0.9118  ms,  for  the  three  principal 
polarizabilities  were  (-0.497,  -0.438,  -0.321),  (-0.380,  -0.270,  -0.150),  (-0.355,  -0.251,  -0.139) 
for  UXO  training  data,  with  a  median  loge  vector  magnitude  of  7.12,  and  were  (-0.682,  -0.444,  - 
0.198),  (-0.369,  -0.203,  -0.0724),  (-0.178,  -0.0815,  -0.0292)  for  the  scrap  training  data,  with  a 
median  loge  vector  magnitude  of  3.20,  where  major  principal  polarizabilities  were  grouped 
together,  intermediate  principal  polarizabilities  together,  and  minor  principal  polarizabilities 
together.  Thus,  the  scrap  data  had  magnitudes  typically  50  times  smaller  than  the  UXO  data,  had 
more  steeply  dipping  polarizability  curves,  and  greater  separation  between  major  and 
intennediate  principal  polarizabilities,  and  much  greater  separation  between  intermediate  and 
minor  principal  polarizabilities.  Median  absolute  deviations  (MADs)  and  a  median  loge  vector 
magnitude  for  the  UXO  were  (0.0249,  0.0225,  0.0140),  (0.0222,  0.0213,  0.0158),  (0.0230, 
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0.0201,  0.0171),  (0.361),  and  for  the  scrap  data  (0.0828,  0.0801,  0.0717),  (0.103,  0.0878, 
0.0518),  (0.106,  0.0585,  0.0245),  (0.720).  These  values  illustrate  two  to  five  times  more 
variability  in  scrap  responses  compared  to  UXO  responses.  Medians  and  MADs  for  the  two 
classes  of  data  with  individual  responses  removed  for  computation  of  densities  using  equation 
(2)  would  be  similar,  but  are  too  numerous  to  consider  here. 
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5.  PERFORMANCE  ASSESMENT 


The  ground  truth  at  San  Luis  Obispo  contained  59  TOI  and  412  pieces  of  scrap.  From  the  total  of 
539  cued  locations  that  BUD  occupied,  66  were  the  training  data,  so  the  number  of  flags  to  be 
discriminated,  using  the  empirical  likelihood  ratio  approach,  was  473,  and  that  was  the  number  of 
entries  in  our  priority  dig  list.  Items  1-154  were  classified  as  Category  1  -  Can  Analyze  -  Likely 
Clutter  and  should  not  be  dug,  and  items  155-473  should  be  dug.  Items  155-281  were  classified 
as  Category  2  -  Can  Analyze:  Cannot  Decide,  items  282-371  were  classified  as  Category3  -  Can 
Analyze:  Likely  Munition,  and  items  372-473  were  classified  as  Category  4  -  Cannot  Analyze. 
Scoring  results  from  the  Institute  for  Defense  Analyses  (IDA)  are  shown  in  Figure  10.  The  ROC 
curve  shows  relationship  between  percents  of  munitions  dug  and  number  of  unnecessary  digs. 
With  154  items  identified  as  scrap,  the  TOI  that  was  missed  in  this  analysis  was  2.36-in  mortar 
under  flag  444.  The  case  of  discrimination  at  the  flag  444  is  particularly  interesting  because,  in 
this  case,  discrimination  results  depend  on  whether  one-dipole  or  two-dipole  polarizability 
inversion  results  were  used  for  the  discrimination.  For  the  one-dipole  inversion  the  response  was 
outside  of  the  typical  range  of  values  in  the  UXO  training  data,  and  the  object  was  classified  as 
scrap.  For  the  two-dipole  inversion,  one  of  the  objects  was  correctly  identified  as  UXO  (puxo 
=0.84),  and  the  other  one  was  correctly  identified  as  scrap  metal. 
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Figure  10:  ROC  curve  for  the  empirical  likelihood  ratio  approach. 
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In  addition  to  the  empirical  likelihood  ratio  approach  we  submitted  also  a  template  match  priority 
dig  list.  In  this  approach,  polarizability  curves  were  matched  with  library  UXO/scrap  responses, 
and  it  contained  only  ‘dig’/’nodig’  decision  for  each  flag.  Since  a  single  object  inversion  was 
used  to  produce  these  responses,  in  case  it  was  not  clear  that  only  single  object  is  present,  if  the 
polarizability  response  at  early  times  was  above  50%  of  the  smallest  TOI  (60  mm  mortar) 
response,  the  flag  was  labeled  as  ‘dig’.  Based  on  this  approach  items  1-  272  could  stay  in  the 
ground,  and  items  273-  473  should  be  dug.  Scoring  results  from  IDA  are  shown  in  Figure  11. 
While  it  appeared  that  using  this  approach  we  missed  2.36-in  mortar  under  the  flag  241/1475, 
further  analysis  and  discussions  showed  that  this  was  a  surface  item,  and  it  was  moved  during  the 
survey,  thus  the  object  was  not  there  when  we  took  the  measurements. 
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Figure  11:  ROC  curve  for  the  template  match  approach. 


Using  the  empirical  likelihood  ratio  approach,  -20%  of  data  were  in  Cannot  Analyze  category, 
95%  of  all  TOI  were  correctly  identified  with  only  two  false  positives,  while  it  took  -140  dry 
holes  for  the  remaining  three  TOI  to  be  found.  The  template  match  approach  showed  that  more 
than  50%  of  total  number  of  digs  could  be  saved  while  correctly  identifying  all  TOI.  This 
approach,  however,  needs  to  be  automated  and  produce  all  supporting  statistics  before  it  would 
be  practical  to  use  it  in  the  field. 
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8.  ACRONYMS 


BUD  Berkeley  UXO  Discriminator 

ESTCP  Environmental  Security  Technology  Certification  Program 

FPGA  Field  Programmable  Gate  Array 

GPS  Global  Positioning  System 

IDA  Institute  for  Defense  Analyses 

LBNL  Lawrence  Berkeley  National  Laboratory 

MAD  Median  Absolute  Deviations 

OSSEPP  Off-Site  Safety  &  Environmental  Protection  Plan 

RTK  Real  Time  Kinematic 

QA/QC  Quality  Assurance/Quality  Control 

SERDP  Strategic  Environmental  Research  and  Development  Program 

TOI  Target  of  interest 

UXO  Unexploded  Ordnance 

YPG  Yuma  Proving  Ground 
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